<html>
<head><meta charset="utf-8"><title>integration-test-only-apis · wg-secure-code · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/index.html">wg-secure-code</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/integration-test-only-apis.html">integration-test-only-apis</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="148340498"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/integration-test-only-apis/near/148340498" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> briansmith <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/integration-test-only-apis.html#148340498">(Nov 26 2018 at 01:32)</a>:</h4>
<p>We need a mechanism to expose APIs to integration tests (not within-module unit tests, but the tests under tests/ parallel to src/) that aren't publicly exposed elsewhere. <span class="user-mention" data-user-id="132721">@Tony Arcieri</span> gave one use case in integer-overflow (mocking APIs) and also kindly demonstrated the flaw in the most common workaround for this missing feature: making such APIs available only in debug mode, and running tests only in debug mode, not <code>test --release</code>.</p>



<a name="148340503"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/integration-test-only-apis/near/148340503" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> briansmith <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/integration-test-only-apis.html#148340503">(Nov 26 2018 at 01:33)</a>:</h4>
<p>I also have a need for such test-only APIs, not only exposed to my own integration tests, but also the integration tests to the users of my crates.</p>



<a name="148340510"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/integration-test-only-apis/near/148340510" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> briansmith <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/integration-test-only-apis.html#148340510">(Nov 26 2018 at 01:33)</a>:</h4>
<p>In particular, it is very convenient to be able to swap out a a good PRNG for a deterministic one for fuzzing, known-answer tests, and other tests, but we don't generally want the PRNG to be swapped out in non-test situations.</p>



<a name="148361648"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/integration-test-only-apis/near/148361648" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> brycx <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/integration-test-only-apis.html#148361648">(Nov 26 2018 at 11:31)</a>:</h4>
<blockquote>
<p>In particular, it is very convenient to be able to swap out a a good PRNG for a deterministic one for fuzzing, known-answer tests, and other tests, but we don't generally want the PRNG to be swapped out in non-test situations.</p>
</blockquote>
<p>In case you didn't know about honggfuzz, it has an attribute for exactly that: <a href="https://github.com/rust-fuzz/honggfuzz-rs#conditional-compilation" target="_blank" title="https://github.com/rust-fuzz/honggfuzz-rs#conditional-compilation">https://github.com/rust-fuzz/honggfuzz-rs#conditional-compilation</a></p>



<a name="148362145"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/integration-test-only-apis/near/148362145" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/integration-test-only-apis.html#148362145">(Nov 26 2018 at 11:42)</a>:</h4>
<p><span class="user-mention" data-user-id="133214">@briansmith</span> you probably thought of feature flags, but I'm still curious why they are not good enough?</p>



<a name="148371426"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/integration-test-only-apis/near/148371426" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Tony Arcieri <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/integration-test-only-apis.html#148371426">(Nov 26 2018 at 14:46)</a>:</h4>
<p>in my case, I'm using feature flags, but there's a feature I want to make sure is <strong>never</strong> used in a <code>release</code> build</p>



<a name="148387105"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/integration-test-only-apis/near/148387105" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> briansmith <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/integration-test-only-apis.html#148387105">(Nov 26 2018 at 18:32)</a>:</h4>
<p>Just like Tony, I want to make sure the feature is never used in non-test situations. The whole point of the API is to be secure and this extension point totally breaks the guarantee of security.</p>



<a name="148388411"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/integration-test-only-apis/near/148388411" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zach Reizner <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/integration-test-only-apis.html#148388411">(Nov 26 2018 at 18:48)</a>:</h4>
<p><span class="user-mention" data-user-id="133214">@briansmith</span> if the integration test feature is enabled, you could print out warnings (so a human can see), kill the current process after a short time (so a headless daemon can't last very long), or refuse to run if the current environment isn't a dev environment (e.g. current directory tree has a Cargo.toml, current process tree has <code>cargo</code>).</p>



<a name="148403312"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/integration-test-only-apis/near/148403312" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Shnatsel <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/integration-test-only-apis.html#148403312">(Nov 26 2018 at 22:27)</a>:</h4>
<blockquote>
<p>In case you didn't know about honggfuzz, it has an attribute for exactly that: <a href="https://github.com/rust-fuzz/honggfuzz-rs#conditional-compilation" target="_blank" title="https://github.com/rust-fuzz/honggfuzz-rs#conditional-compilation">https://github.com/rust-fuzz/honggfuzz-rs#conditional-compilation</a></p>
</blockquote>
<p>Actually that's supported by all three Rust fuzzers: cargo-fuzz, <a href="http://afl.rs" target="_blank" title="http://afl.rs">afl.rs</a>, honggfuzz-rs.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>